Skip to content

feat: add JBOD additional PVC support for ClickHouse replicas#136

Open
matanper wants to merge 8 commits intoClickHouse:mainfrom
matanper:jbod
Open

feat: add JBOD additional PVC support for ClickHouse replicas#136
matanper wants to merge 8 commits intoClickHouse:mainfrom
matanper:jbod

Conversation

@matanper
Copy link

Introduce additionalDataVolumeClaimSpecs with validation, templating, and reconciliation support so replicas can mount and configure multiple persistent disks while safely updating per-template PVC specs without immutable StatefulSet failures.

  • Add spec.additionalDataVolumeClaimSpecs to ClickHouseCluster so each replica can attach multiple data PVCs for JBOD layouts.
  • Generate and mount additional PVCs in StatefulSets, and render JBOD storage config (10-storage-jbod.yaml) for ClickHouse.
  • Add webhook validation for additional volume names/immutability constraints.
  • Reconcile PVC changes per volumeClaimTemplate name to avoid immutable StatefulSet update failures when PVC specs change.
  • Include tests and an example manifest (examples/multi_disk_jbod.yaml).

Why

ClickHouse multi-disk (JBOD) deployments currently require workarounds and do not have first-class support in the operator API. This change adds explicit support for attaching additional per-replica PVCs and configuring them in ClickHouse, while preserving safe reconciliation behavior with StatefulSet immutability constraints.

What

This PR adds a new ClickHouseCluster field, spec.additionalDataVolumeClaimSpecs, to define extra per-replica PVC templates (name,

Introduce additionalDataVolumeClaimSpecs with validation, templating, and reconciliation support so replicas can mount and configure multiple persistent disks while safely updating per-template PVC specs without immutable StatefulSet failures.
- Use a single volume with all disks for true JBOD round-robin
  distribution instead of one volume per disk (which was tiered storage)
- Remove redundant MountPath fallback in storageJbodConfigGenerator.Generate;
  WithDefaults() already guarantees a non-empty value
- Validate duplicate mountPaths across additionalDataVolumeClaimSpecs,
  including implicit defaults colliding with explicit paths
Remove dead ReplicaUpdateInput.DataVolumeClaimSpec field, rename
primaryPVCName to targetPVCName for clarity, always preserve
immutable volumeClaimTemplates on StatefulSet updates, fix potential
panic on empty MountPath, and extract AdditionalDiskBasePath constant.
Truncate the generated version probe Job name prefix so the final name stays within the 63-character Kubernetes label value limit, and add focused unit tests for truncation and non-truncation cases.
Handle additional disk updates for existing ClickHouse clusters by creating per-replica PVCs separately from StatefulSet volumeClaimTemplates, updating pod mounts to use explicit PVC volumes, and allowing additive additionalDataVolumeClaimSpecs updates while blocking remove/rename operations.
ReconcilePVC previously delegated to reconcileResource which replaced the
entire PVC spec, causing API rejections on immutable fields (VolumeName,
VolumeMode, AccessModes, StorageClassName, etc.) for bound PVCs.

Replace with a surgical merge patch that only touches the actually mutable
fields: spec.resources.requests.storage, spec.volumeAttributesClassName,
and labels. StorageClassName divergence is detected and surfaced as a
warning event rather than silently ignored or sent as an invalid update.
…lickHouse compatibility

Disk names used in additionalDataVolumeClaimSpecs serve dual purposes:
as Kubernetes PVC/volume names (requiring lowercase alphanumeric + hyphens)
and as ClickHouse disk identifiers (forbidding hyphens). Previously there
was no format validation, allowing names like disk-backfill-1 to pass
admission but fail in ClickHouse, or names with underscores to pass but
fail Kubernetes PVC creation.

- Add webhook validation enforcing Kubernetes DNS label rules
  (lowercase alphanumeric and hyphens, must start/end with alphanumeric)
- Add matching +kubebuilder:validation:Pattern marker to the CRD type so
  the API server enforces the same constraint even without the webhook
- Regenerate CRD manifests
- Sanitize hyphens → underscores in the ClickHouse JBOD config generator
  so users use Kubernetes-valid names and the operator handles the
  ClickHouse naming requirement transparently
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant